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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .1 36(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 133). 

- Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

1 )^ Responsive to communication(s) filed on 24 August 1999 . 
2a)\3 This action is FINAL. 2b)S This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quay/e, 1935 CD. 1 1 , 453 O.G. 213. 
Disposition of Claims 

4) ^ Claim(s) 1-52 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) [3 Claim(s) 1-52 is/are rejected. 

7) Q Claim(s) is/are objected to. 

8) 0 Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)^ The drawing(s) filed on 26 April 1999 is/are: a)D accepted or b)S objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
1 1 )□ The proposed drawing correction filed on is: a)D approved b)D disapproved by the Examiner. 

If approved, corrected drawings are required in reply to this Office action. 

12) D The oath or declaration is objected to by the Examiner. 
Priority under 35 U.S.C. §§ 119 and 120 

13) ^ Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 

a)D AM b)D Some*c)Q None of: 

1 Certified copies of the priority documents have been received. 

2. n Certified copies of the priority documents have been received in Application No. . 

3. ^ Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 1 7.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 

14) D Acknowledgment is made of a claim for domestic priority under 35 U.S.C. § 1 19(e) (to a provisional application). 

a) □ The translation of the foreign language provisional application has been received. 

15) Q Acknowledgment is made of a claim for domestic priority under 35 U.S.C. §§ 120 and/or 121. 
Attachment(s) 

1) Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) Paper No(s). . 

2) Notice of Draftsperson's Patent Drawing Review (PTO-948) 5) D Notice of Informal Patent Application (PTO-152) 

3) □ Information Disclosure Statement(s) (PTO-1449) Paper No(s) . 6) □ Other: 



U.S. Patent and Trademark Office 
PTO-326 (Rev. 04-01) 



Office Action Summary 



Part of Paper No. 6 



Application/Control Number: 09/297,038 
Art Unit: 2654 



Page 2 
Paper #6 



1. Applicant's correspondence filed on 24 August 1999 (paper #4) has been received and 
considered under 35 USC 371 as indicated by the Form PCT 903 mailed 22 Sep 1999 (paper #5). 
Claims 1-52 are pending. 



2. The Abstract of the Disclosure is objected to because it is confusing. It is not clear 
whether a musical number is merely a designation or a actual musical work. The definition of 
"karaoke information" is unclear and it is also unclear whether the device generates synthesized 
vocal output or whether a human being is doing this. Correction is required. See M.P.E.P. 
§ 608.01(b). 



3. The drawings are objected to under 37 CFR 1.83(a). The drawings must show every 
feature of the invention specified in the claims. Therefore, the information that is being 
processed must be shown or the feature(s) canceled from the claim(s). For example, the "lyric", 
"accompaniment", "language letter" and "synthesized information." The drawings fail to show 
how any type of data separation is performed. The drawings must show the data format and data 
structure relied upon as well as the method for processing the data to achieve the desired result 
commensurate with the description and claims. 
No new matter should be entered. 

A proposed drawing correction or corrected drawings are required in reply to the Office 
action to avoid abandonment of the application. The objection to the drawings will not be held 
in abeyance. 



Abstract 



Drawines 
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Claims 

4. Claims 1-52 are rejected under 35 U.S.C. § 1 12, second paragraph, as being indefinite for 
failing to particularly point out and distinctly claim the subject matter which applicant regards as 
the invention. 

Claims 1, 13, 24, 31 and 45 are rejected as noted below. 

The claims are confusing because they fail to specify what type of information they 
intend to process. The input is not defined nor is the manner of "separating the lyric information 
part and an accompaniment information part from the input information." 

"Generating the first language letter information by speech recognition of the 
lyric" indicates that someone speaks the lyrics into the device. From the antecedent reference of 
the separation unit, the lyrics must be derived from speech. Therefore, the accompaniment 
should also be limited to speech. However, this contradicts the specification such as page 16, 
lines 8-12 which describes the input/output relationship such that the data is already stored in a 
separated format "The storage unit 212 separates the musical number information . . . into the 
vocal part information (vocal information) and the accompaniment part information other than 
the vocal part (karaoke information) to output the separated information." 

The claims are broad enough to include the separation of any speech from any input 
audio. For example, this would include choral music and the separation of a typical 4 part 
(SATB) harmony into the desired lyrics of one or more parts (the lyrics could vary among parts). 
Other possibilities would include one or more singers accompanied by a band or orchestra in 
which separation of parts could be much more complicated between singers, instruments and/or 
accompaniment parts. 
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The claimed "language letter information" in combination with a " separation unit" and 
"processing unit" described in the specification as "storage unit 212 separates. . ." implies that the 
data is in a predefined format which was separated into desired portions prior to transmission. 

It is unclear whether the applicant intends to limit the invention to musical related 
information or whether the combination of data can include a wider variety of multimedia 
information. Therefore, the separation of data is interpreted to be broad enough to include 



various types of information known in the art. 

The only specific application mentioned in the specification is for karaoke related data. 
However, neither the specification nor the claims clearly describe any particular requirements for 
data structure or data format for karaoke devices and/or methods for using karaoke devices. 

5. The following is a quotation of the first paragraph of 35 U.S.C. 1 12: 

The specification shall contain a written description of the invention, and of the manner and process of making 
and using it, in such full, clear, concise, and exact terms as to enable any person skilled in the art to which it 
pertains, or with which it is most nearly connected, to make and use the same and shall set forth the best mode 
contemplated by the inventor of carrying out his invention. 

6. Claims 1-52 are rejected under 35 U.S.C. 112, first paragraph, as containing subject 
matter which was not described in the specification in such a way as to enable one skilled in the 
art to which it pertains, or with which it is most nearly connected, to make and/or use the 
invention. 

The specification on page 6 indicates that "the required information" is not particularly 
limited but may include various data "such as audio information, text information, image 
information or the picture information as later explained..." The specification fails to limit the 
information to any particular format or combination of data. This makes it unlikely that one of 
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ordinary skill in the art could hope to predict how to process the data. The desired result of the 
processed data is similarly vague. 

The transmission method is not limited to any particular format. On page 8, the applicant 
states: "There is no particular limitation to the communication network 4, [figures 1 and 3] such 
that it is possible to utilize CATV (cable television, community antenna television), 
communication satellite, public telephone network or wireless communication. . Therefore, the 
method of transmission is all inclusive and does nothing to define the data or its components. 

Page 9 of the specification describes "the intermediate transmission devices 2" in a 
similarly generic fashion. Figure 3 shows elements of device 2 but the description is similarly 
vague. Page 9, last paragraph indicates that devices 2 may be anywhere and are made up of "a 
display unit 203 for optionally displaying the required contents associated with the operations 
and a key actuating unit 202." Page 10 merely indicates that device 2 "is also provided with a 
terminal device attachment portion 204 for attaching the portable terminal device 3... while the 
power supply terminal 206 is electrically connected to a power input terminal 307 of the portable 
terminal device 3." Page 11 merely indicates that these generic connections allow transmission 
of data and necessary power to both devices 2 and 3. 

On page 12, last paragraph, the server device is described in part as containing "an 
assessment processing unit 105 for assessment processing for the user and an interfacing unit 106 
for having communication with the intermediate transmission device 2." The function of the 
"assessment processing unit 105" is undefined. Neither description of the data being assessed 
nor any description of the resultant assessment is provided to give life and meaning to these 
terms. 
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On page 13, second paragraph, the applicant states: "a unique protocol or TCP/IP 
(Transmission Control Protocol/Internet Protocol) transmitting data generally used on the 
Internet by packets, may be used." This indicates that the applicant may employ an undefined 
"unique protocol" or a standard protocol such as TCP/IP. Because the data necessary to the 
invention is not clearly defined, the reader is unable to determine whether a "unique protocol" 
must be proprietary to achieve desired results or whether a standard protocol could really be used 
to achieve the same desired results. Even more problematic is that the desired results are 
unknown making further analysis virtually impossible. 

Applicant improperly relies upon foreign [Japanese] documents H-3- 139923 or 3-13922 
for teaching how to make and use TwinVQ. The specification must fully explain TwinVQ by 
including the necessary text from these documents or should remove all mention of this 
technique. If this is a trademark, then, it must additionally be placed in all capitals. 

Page 14, second full paragraph fails to indicate what data is "collated". Supposedly, 
"The terminal ID data of the portable terminal device 3" is magically collated with "the terminal 
ID data of the portable terminal device that is currently able to use the information distribution 
system". How is a single device "currently able to use the information..." identified? Since 
page 13 implies use of the Internet, why would only one device (such as a computer) be able to 
use such information? The last sentence of this paragraph has grammatical errors that the 
Examiner cannot resolve to gain a reasonable understanding of its intended meaning. It is 
unclear how a device can be physically loaded onto another device and the use of such a device 
or system is confusing. 

The last paragraph of page 14 (continuing to page 15) does not explain what sort of 
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"assessment" is desired. The sentence "... the assessment processing unit 105 performs the 
processing of assessment of the amount in meeting with the state of use of the information 
distribution system by the user. . ." is incoherent. Neither the data being assessed, the process by 
which assessment is performed, what "amount" (amount of what?) is utilized nor the "state of 
use" are defined. The applicant throws about these terms without being given any meaningful 
definitions thereof. The example given is nonsense: "the request information for information 
copying or electrical charging. . ." How can a request for "information copying" be treated as an 
alternative to "electrical charging" (charging a battery?). What does each of these things really 
mean? 

The functionality of the "key actuating device 202" is unknown. On page 15, it is 
indicated that this is a necessary part of an "intermediate transmission device 2" and that the 
actuating device 202 must be "actuated by a user." However, page 9 (last 2 lines) indicates that 
unit 2 may use a "display 203" to optionally display "required contents associated with the 
operations and a key actuating unit 202." Page 11 (last paragraph) indicates that units 202 and 
203 may be omitted. The only portion that really looks like it contains keys capable of actuation 
by a user is 302. Some keys in 302 appear to be standard rewind, play, fast-forward, record and 
stop functions but the others are unknown and undefined. The functionality of 202 is confusing 
because it looks like a display but is described as a key. 

A "storage unit 212" described on page 16 (third paragraph) is not capable of performing 
a separation function. This is probably a typing error. However, no evidence that the applicant's 
invention can take a song and separate portions of the vocal information and accompaniment 
(vocal and instrumental) is presented by the applicant. 
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Page 18, second paragraph mentions "speech recognition translation unit 321" and 
"speech synthesis unit 322" but provides no details capable of actually achieving the desired 
results of either unit. 

Page 19, last paragraph, indicates that "speech recognition translation unit 321 is fed with 
the vocal information transmitted along with the karaoke information after separation by the 
vocal separation unit 212 of the intermediate transmission device 2, and performs speech 
recognition of the vocal information. Again, no details for separating desired audio (a particular 
vocal) from other audio data is even offered by the applicant. Similarly, no details for 
performing speech recognition and translation to another language are offered. As a minimum, 
the drawings should show the steps of analysis necessary to input typical song data and extract 
specific parameters that can be analyzed by a computer to determine the desired results. Details 
must be provided giving a reasonably detailed explanation of how one of ordinary skill in the art 
could expect successful separation, recognition and translation. 

Page 20, first full paragraph, indicates that "speech synthesis unit 322 first generates the 
novel vocal information (audio data) sung with the lyric of the as-translated second language, 
based on the second language lyric information generated by the speech recognition translation 
unit 321. No details for performing such a desired manner of synthesis are provided. The 
apparatus and method for analysis as well as the parameters for modeling "original vocal 
information" must be provided. Similar details for synthesizing speech with musical properties 
must be shown that will allow utilization of "original vocal information." Further details are 
necessary to show "original vocal information" specifics with regard to music and speech. Such 
details must include time and frequency as it relates to both musical pitch and vocal tract and/or 
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other information that is specific to language idiosyncrasies. For example, in English, changes in 
pitch do not change the literally meaning of a word, but in certain Easter languages (i.e. - 
Chinese) a rising or falling pitch could change the meaning of an otherwise identical 
pronunciation. Such details provide interesting challenges to the desired results of applicant's 
invention. However, no details are provided to address these or even more basic information. 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or 
described as set forth in section 102 of this title, if the differences between the subject 
matter sought to be patented and the prior art are such that the subject matter as a whole 
would have been obvious at the time the invention was made to a person having ordinary 
skill in the art to which said subject matter pertains. Patentability shall not be negatived 
by the manner in which the invention was made. 

8. Claims 1-52 are rejected under 35 U.S.C. § 103 as being unpatentable over Stelovsky 
(5,613,909) in view of Bordeaux (4,852,170) and Lyberg (5,546,500). 

As per claim 1, "information processing" is taught by both references: 

"separating the lyric information part and an accompaniment part from the input" 
(Stelovsky teaches the separation of lyrics in figure 5); 

"translating the generated first language letter information into the second 
language letter information" (suggested in column 14, lines 21-22 using direct translation into 
another language ); and 
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synthesizing the speech information" (suggested in column 14, lines 18- 



19 where he teaches that the audio track can be generated rather than recorded (e.g. using a 
speech generator.) ). 

It is noted that Stelovsky does not explicitly teach the use of "speech recognition" to 
perform translation. However, he teaches that translation is obvious in combination with a 
karaoke or other multimedia separation of data elements in order to facilitate education and/or 
entertainment. Bordeaux and Lyberg teach details for performing speech recognition and in 
column 12, lines 60-65, Bordeaux teaches that for use in foreign languages ... a different natural 
language or orthographic translator would be employed . It would have been obvious for a 
person having ordinary skill in the pertinent art, at the time the invention was made, to combine a 
speech recognition based translator such as Bordeaux with the device of Stelovsky because 
Stelovsky specifically invites the use of future facilities (col. 14, line 11) which include 
translation into other languages as noted above. Lyberg explicitly recites Bordeaux in column 1, 
line 24 and is utilized because he clearly teaches that it is known to combine synthesis with a 
translation device (see abstract) in such a way as to preserve prosodic information even after 
translation. 

Claims 2-52 are rejected under similar arguments as presented above. Although the 
claims are unclear, it is presumed that the applicant is attempting to limit some of the synthesis 
related elements to preserving information gathered during analysis or recognition. This is 
taught by Lyberg who preserves prosody following recognition and translation for use in 
synthesis. 
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Prior Art 



9. 



The prior art made of record and not relied upon is considered pertinent to applicant's 



disclosure. 



Stelovsky (5,782,692) provides cumulative evidence over Stelovsky (5,613,909). 

10. Any response to this action should be mailed to: 

Commissioner of Patents and Trademarks 
Washington, D.C. 20231 

or faxed to: 

TC2600 Fax Center 
(703) 872-9314 

Hand-delivered responses should be brought to Crystal Park II, 2121 Crystal Drive, Arlington. 
V A., Sixth Floor (Receptionist). 

11. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David D. Knepper whose telephone number is (703) 305-9644. 
The examiner can normally be reached on Monday-Thursday from 07:30 a.m.-6:00 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Marsha Banks-Harold, can be reached on (703) 305-4379. 

Any inquiry of a general nature or relating to the status of this application should be 
directed to customer service whose telephone number is (703) 306-0377. 




David D. Knepper 
Primary Examiner 
Art Unit 2654 
July 25, 2002 



